Bugzilla:
https://bugzilla.mozilla.org/show_bug.cgi?id=2055853

Add Performance Regression Predictor (Autoland push-based inference)

Summary

Adds an inference-only model, an HTTP service endpoint, and a CLI that predict
whether a commit is likely to introduce a performance regression. The
service takes an Autoland/try push identified by (branch, rev) — the same
contract as the test-selection endpoint — and resolves everything server-side
from its own local Mercurial clone. Each commit in the push is scored
independently; the push-level risk is the max across commits.

Motivation

Give CI/taskgraph an early signal for pushes that are likely to regress
performance, without plumbing diffs or credentials from Phabricator. Reusing the
(branch, rev) + local-clone approach that schedule_tests already uses keeps
the integration surface minimal and consistent.

How it works

The worker (classify_perf_regression(branch, rev)) mirrors schedule_tests:

1. repository.pull(REPO_DIR, branch, rev) into the service's local hg clone.
2. get_hgmo_stack(branch, rev) loads the full push stack via
   automationrelevance (with try's draft/empty/try-config filtering).
3. Each commit is exported from the clone (hg export --git) to get its message
   and diff.
4. Each commit is scored one at a time (CPU inference — avoids multiplying
   peak memory by stack size).
5. The response contains per-commit scores plus the max as the top-level
   risk_score.

The model is a HuggingFace CodeBERT sequence classifier (inference-only,
512-token window, binary). The input is the cleaned commit message plus the diff
rendered into a structured <FILE>/<ADDED>/<REMOVED> representation matching
training. The risk_score is the raw (uncalibrated) softmax probability of the
positive class.

API

GET /perfregressionpredictor/predict/push/<branch>/<rev>
X-Api-Key: ...

branch is an hg.mozilla.org path (integration/autoland, try; autoland is
accepted as an alias). The first call returns 202 {"ready": false}; poll until
200. A missing push returns {"available": false}.

Example result:

{
  "branch": "integration/autoland",
  "rev": "76383a875678",
  "risk_score": 0.75,
  "commits": [
    {"node": "…", "prob": [0.25, 0.75], "class": 1, "risk_score": 0.75}
  ],
  "extra_data": {
    "model_name": "Perf Regression Predictor",
    "max_length": 512,
    "calibrated": false,
    "commit_count": 1
  }
}

Also included

• CLI (bugbug-predict-perf-regression) for local inference against a
  checkpoint + patch file, with no service/Redis/clone required.
• Model distribution: the model is trained outside bugbug, so it declares a
  public artifact_url; download_models now supports fetching a model from a
  URL instead of the Taskcluster index.
• Dependencies: new optional perf-regression-predictor extra
  (torch==2.13.0, transformers==5.15.0, pinned to the CPU wheel index).
• Docs: model docs (docs/models/perf-regression-predictor.md) and the
  HTTP-service local-dev guide
  (http_service/README.perf-regression-predictor.md).

Testing

Unit test:

cd /home/ali/repos/bugbug
python -m pytest tests/test_perf_regression_predictor.py http_service/tests/test_perf_regression_predictor.py

Via docker compose:

cd /bugbug/http_service

docker compose up --build \
  redis \
  bugbug-http-service \
  bugbug-http-service-bg-worker

docker compose exec bugbug-http-service-bg-worker \
  python -c "from bugbug import utils; from bugbug.models.perf_regression_predictor import PerfRegressionPredictorModel as M; utils.download_model_from_url('perfregressionpredictor', M.artifact_url)"


docker compose exec bugbug-http-service-bg-worker \
  test -f /code/perfregressionpredictormodel/config.json \
  && echo "Model is mounted"


docker compose logs -f bugbug-http-service-bg-worker   # wait for "Worker boot done"


curl --compressed -sS -w '\nHTTP %{http_code}\n' \
  -H "X-Api-Key: local-test" \
  "http://localhost:8000/perfregressionpredictor/predict/push/autoland/540bd3cda62fd69f9e38ed8053d938a5aeed21c0"


docker compose down